Teaching Stereo Perception to YOUR Robot

This paper describes a method for generation of dense stereo groundtruth using a consumer depth sensor such as the Microsoft Kinect. Such ground-truth allows adaptation of stereo algorithms to a specific setting. The method uses a novel residual weighting based on error propagation from image plane measurements to 3D. We use this ground-truth in wideangle stereo learning by automatically tuning a novel extension of the best-first-propagation (BFP) dense correspondence algorithm. We extend BFP by adding a coarse-to-fine scheme, and a structure measure that limits propagation along linear structures and flat areas. The tuned correspondence algorithm is evaluated in terms of accuracy, robustness, and ability to generalise. Both the tuning cost function, and the evaluation are designed to balance the accuracy-robustness trade-off inherent in patchbased methods such as BFP. Wide-angle stereo provides an overview of a scene, even at very short range. The large field of view (FoV) also ensures that the visual fields from different points of view have a high degree of overlap. For these reasons, wide-angle lenses are popular in navigation, mapping and visual object search on robot platforms. However, the radial distortion caused by these lenses complicates the application of traditional stereo algorithms. A common approach to wide-angle stereo is to first attempt to remove radial distortion and then apply a descriptor-based wide-baseline stereo algorithm. An alternative approach is to use simpler matching metrics, and instead leverage correspondence propagation. One such algorithm is the best-first propagation (BFP) algorithm [2], and a recent addition is the generalised PatchMatch algorithm (GPM) [1]. Though these algorithms are more general than stereo algorithms, they have previously been applied to the stereo problem. Since we make use of both the inverse depth and pixel coordinates in our calibration procedure, the effect of errors in these measurements must be taken into account during calibration. We therefore propagate error variances from these measurements into both 3D reconstructions and resulting 2D projections. We then fuse multiple Kinect range scans in a reference view coincident with the left stereo camera. This is done by estimating a disparity distribution for each pixel in this image, and using mean-shift to find the visible surface closest to the camera. We have imaged three indoor scenes, and for each of them calculated 51 full-resolution wide-angle disparity maps. Examples of individual range scans, an intermediate point cloud and the final disparity maps are shown in figure 2 (top row). We use these to tune our extension of the BFP algorithm, which we call coarse-to-fine best-first propagation (CtF-BFP). Novelties are the use of multiple scales, a structure threshold that limits propagation along linear structures, and a sub-pixel refinement step. These novelties add a multitude of parameters, which make manual tuning difficult. We therefore use an automatic tuning procedure, that minimises an objective function that balances on accuracy, coverage and robustness. When measuring performance of the stereo algorithm, we denote the estimated disparity map to be evaluated by D(u,v) defined on the domain V (pixels where disparities have been estimated). Similarly, the groundtruth disparity image is D∗(u,v), and the set of valid ground-truth pixels is V∗. To find a useful set of parameters, we minimise an objective function